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A system and device is provided that enhances the interactivity of multimedia information across a communications ne- 
twork or in combination with other networks including telecommunication and cable broadcasting. The system includes a plurali- 
ty of multimedia call processing system (44), an interactive call processing system (42), and a plurality of interactive multimedia 
devices (52). This system has the advantage of providing compression and/or transmission algonthms to maximize enhancement 
of the multimedia information. 
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INTERACTIVE MULTIMEDIA COMMUNICATION SYSTEM 

FIELD OF THE INVENTION 

The present invention relates to a method and system 
for the production, transmission and reception of 
multimedia information over a communication network and 
more particularly to the production, transmission and 
reception of multimedia information with enhanced 
interactivity over such a network. 
BACKGROUND OF THE INVENTION 

Multimedia information can be transmitted in a 
variety of applications over a network. What is meant by 
multimedia information is information that has several 
parameters which can be enhanced or diminished depending 
upon the importance of the particular information being 
conveyed. For example, in a video environment there may 
be foreground and background information that is to be 
provided to a screen. The screen can typically be a 
video screen or a computer screen. The most common 
example of that type of system is an interactive video 
system such as seen on cable television networks (i.e., 
QVC or Home Shopping Network) . These types of systems 
allow for a viewer to look at the screen and pick the 
type of product that he/she may want based on reviewing 
the information on the screen and then calling a 
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particular telephone number to purchase the product. 

Multimedia information can also be audio information 
in which news, weather, music and like could be 
transmitted with the use of background information such 
as white noise, background music or other information to 
complete the transmission. Multimedia information can 
also be a combination of graphics, video and music in an 
entertainment form such as Karaoke. As has been above 
mentioned, there are interactive systems presently 
available that allow for individuals to access certain 
video, audio and graphics information on a network. All 
presently known systems have the problem of requiring a 
significant amount of bandwidth to allow for the quality 
transmission of the multimedia information. 

It is known that multimedia information can require 
a significant amount of bandwidth. In the case of video 
information as much as 3 0 MB/ sec. bandwidth is required 
to send a quality video image along a transmission path. 
In the case of audio information, as much as 10 MB/ sec. 
bandwidth is required to allow for high quality audio 
signal to be produced accurately. 

What is meant by limited bandwidth can be viewed in 
two different ways. One way is to view the total 
available bandwidth at any instant in time. Another way 
of deciding what the particular bandwidth requirement is 
by determining the bandwidth that is available over a 
specified time period. 
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The available bandwidth therefor. can be limited in 
two ways; first, the overall bandwidth of the network is 
limited such as in telephone networks where the- bandwidth 
is approximately ten (10) kilohertz. However in this 
example, the available bandwidth over a specified period 
of time could be very high because a particular telephone 
set is not utilized constantly. 

On the other hand, in looking at a large bandwidth 
network such as cable television, although the bandwidth 
is significant, most of the available bandwidth is 
utilized for providing the network signals. In such an 
example, the available bandwidth over a specified period 
of time might be much less than the above-mentioned 
telephone system. 

Telephone networks have traditionally been used to 
transmit data or voice information. There are telephone 
sets that can also transmit video information over 
telephone lines. For example, American Telephone & 
Telegraph (AT&T) has introduced a phone called the 
Videophone which can transmit video images across the 
telephone network. It is becoming readily apparent that 
the telephone networks can be used for interactive 
multimedia information also. However, as has been 
mentioned above a typical telephone network has a limited 
bandwidth to transmit certain information. Particularly 
when attempting to transmit video information which might 
require significant bandwidth the telephone lines are 
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inadequate to provide quick transmission of a high 
quality video image. 

In addition, in such systems the communication 
network, such as a telephone network itself, will not 
provide the enhancement capabilities required to provide 
the psychographic information that will enhance 
interactivity. Finally, in a typical telephone network 
many of the technologies required for such a system has 
not entered the home. For example, although there are 
many known video enhancing capabilities, audio enhancing, 
printing, telephone call processing control systems, and 
the like in commercial use, those techniques are not 
typically part of the home environment. 

Hence, what is needed is an interactive multimedia 
system that provides high quality information over a 
telephone network. What is also required is a system 
that will allow for the production, transmission, and 
reception of interactive multimedia information that 
includes enhanced interactivity. What is also needed is 
a device that can be utilized within the home that can 
receive and transmit enhanced multimedia information and 
control a number of other devices for useful purposes. 

It is important that such a device be inexpensive to 
install in the home environment. Furthermore, it is 
important that the interactive multimedia device be one 
that when installed does not require the modification of 
the existing network. 
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The present invention provides such a system and 
device to be utilized in conjunction with a communication 
network such as a telephone network or the like.* 
SUMMARY OF THE INVENTION 
5 An interactive multimedia communication system is 

disclosed that can be utilized with a telephone network, 
a similar switched network or in combination with a 
broadcast network such as satellite or cable. 

The interactive multimedia communication system 
10 comprises interactive multimedia mastering system means 

for receiving program materials from a program source; 
multimedia call processing system responsive to 
interactive multimedia mastering system program materials 
for providing information; means interactively responsive 
15 to the interactive multimedia mastering system means and 

the multimedia call processing system for controlling the 
flow of multimedia information to the multimedia call 
processing system; and a plurality of interactive 
multimedia devices for receiving and transmitting 
20 multimedia information to and from the multimedia call 

processing system. 

In another facet of the invention, the interactive 
multimedia device comprises a multimedia modem means for 
receiving interactive multimedia information from the 
25 telephone network; memory means for receiving interactive 

multimedia information from the multimedia modem means; 
means for compressing at least a portion of the 
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interactive multimedia information received from the 
multimedia modem means and the memory means; means for 
decompressing at least a portion of the interactive 
multimedia information; means for transmitting the 
uncompressed portion of the interactive multimedia 
information received from the telephone network; mixing 
the decompressed portion with the uncompressed portion 
and providing an output; means for providing the output 
of the interactive multimedia information to a display; 
and means for interactively controlling the multimedia 
modem means, compressing means, decompressing means, 

mixing means and providing means to provide interactive 

multimedia information to the display. In one facet of 
the present invention there are means to control other 

devices to perform specific functions including printing 

and the control of household devices. 

Through such a system and device a system can be 

utilized with an existing telephone network to produce 

high quality multimedia information. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram of a prior art 

interactive multimedia system. 

Figure 2 is a block diagram of an interactive 

multimedia system for use in a telephone network alone or 

in combination with a cable tv broadcast feed in 

accordance with the present invention. 

Figure 3 is a flow chart of a first embodiment of 
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the optimization method which is utilized in a telephone 
network in accordance with the present invention. 

Figure 4 is a flow chart showing the operation of a 
second embodiment of an optimization method which is 
utilized in a telephone network in accordance with the 
present invention in which a matrix of compression and 
transmission algorithms are utilized. 

Figure 5 is a flow chart showing psychographic 
enhancement in accordance with the present invention. 

Figure 6 is a chart showing the cooperation of a 
compression algorithm with a transmission algorithm in 
accordance with the present invention. 

Figure 7 is a block representation of digital 
information of an image file and a MIDI file. 

Figure 8 is a block representation of the multimedia 
call processing system (MCPS) in accordance with the 
present invention. 

Figure 9 is a block diagram of a general embodiment 
of an interactive multimedia device (IMD) in accordance 
with the present invention. 

Figure 10 is a block diagram of a second embodiment 
of an IMD in which the IMD is connected to a printer in 
accordance with the present invention. 

Figure 11 is a block diagram of a third embodiment 
of an IMD in which the IMD produces a facsimile in 
accordance with the present invention. 

Figure 12 is a block diagram of a fourth embodiment 
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of ah IMD in which the IMD is connected to a facsimile 
machine in accordance with the present invention. 

Figure 13 is a representation of a remote control 
utilized in conjunction with the system architecture of 
the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

The present invention relates to an improvement in 
the transmission and reception of multimedia information 
when utilizing existing networks. The following . 
description is presented to enable one of ordinary skill 
in the art to make and use the invention and is provided 
in the context of a patent application and its 
requirements. Various modifications to the preferred 
embodiment will be readily apparent to those skilled in 
the art and the generic principles and features described 
herein. 

Referring now to Figure 1, what is shown in simple 
block diagram form is a prior art interactive multimedia 
system 10. In such a system 10, a sender sends and 
receives information on one side of system 10 and the 
receiver sends and receives information on the other side 
of the system 10. The system 10 can be a single device 
such as a CD-ROM player or a network capable of fully 
interactive (2-way) multimedia transmission. 

Before the present invention is described in detail 
certain background information should be discussed to 
clarify the utility of the present invention in terms of 
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existing multimedia systems. In a multimedia system, 
various sensory information is provided to a receiver. 
In the case of video information, this information takes 
the form of foreground and background images that display 
a particular scene. In the case of audio information, 
the foreground and background signals are such that the 
foreground information is speech and the background 
information is music. 

Typically, multimedia systems in whatever form 
provide this information over a single transmission line. 
In so doing, the amount and quality of the multimedia 
information is severely limited by the bandwidth of the 
transmission line. 

The present invention first differentiates between 
important and less important multimedia information by 
separating the information into primary and secondary 
layers through the use of a program model to minimize the 
bandwidth limitations. In such a system each layer will 
have its own set of parameters that are important 
psychographically , however, the secondary layer will not 
vary as much as the primary layer. 

What is meant by a program model refers to 
psychographic parameters within the multimedia system, 
that is parameters that relate to an individual's sensory 
perceptions when encountering multimedia information. 
These parameters comprise a set of unique and consistent 
elements for a particular class of multimedia 
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information. In accordance with the present invention, 
its multimedia information is separated into different 
layers in accordance with the program model • Therefore, 
by way of example, in the instance of video images, the 
foreground and background information might be divided 
into different layers. Similarly, in the case of audio 
information, the news information, weather information, 
or the like may be one layer whereas the background music 
may be the other layer. 

In the present invention these layers will be 
divided into primary and secondary layers in accordance 
with the information's importance relative to the program 
model. The most important information is identified and 
enhanced to provide the best quality information to the 
receiver of the multimedia information. 

In the preferred embodiment, the primary layers will 
be enhanced in such a way to provide a perceived 
improvement in quality of the multimedia information 
presented. In one embodiment the secondary layers are 
presented that may or may not be enhanced. Thereby the 
important information or the primary layers that are to 
be transmitted can be identified and selectively enhanced 
in accordance with the present invention. 

In addition, the primary layers generally can be 
enhanced through critical psychographic parameters take 
the form of spatial, color, audio, and temporal variables 
that occur in the primary or secondary layers. 
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In a communication system such as a telephone 
network it is important that the multimedia information 
that is produced, transmitted and received is enhanced in 
some manner. This is necessary to ensure that high 
5 fidelity, high quality information is presented to the 

viewer. Therefore, it will be possible to bring a 
superior product into the home environment. It is known, 
for example, it has been possible to provide video 
information over the telephone lines. However, it has 

10 been a problem sending high quality video information due 

to the bandwidth requirements that are needed to provide 
such high quality video information. 

The present invention is directed towards a method 
and apparatus for enhancing the interactive multimedia 

15 information transmitted across a network. To more fully 

explain such a method and apparatus, refer now to Figure 
2 which is a block diagram of the overall system 
architecture 40 for an enhanced interactive multimedia 
system. The overall architecture 4 0 would be connected 

20 to a telephone network or the like so as to readily 

access the transmission lines located therein. Referring 
now to Figure 2, the system architecture 40 comprises an 
interactive multimedia mastering system (IMM) 42 which 
receives program source material. The IMM system 

25 provides information from the telephone network to a 

multimedia call processing system (MCPS) 44 and an 
auxiliary system 46. The multimedia call processing 
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system 44 will provide and receive information from 
interactive multimedia devices (IMDs) 52. The structure 
of the IMDs 52 will be described in detail later in this 
specification. These IMDs 52 can take on many different 
forms dependent upon how much intelligence is located in 
an IMD 52 relative to the system architecture 40. 

The system architecture 40 provides program material 
which will enhance the interactivity of information that 
is transmitted along the telephone network. Accordingly, 
what the system architecture 4 0 represents, in fact, is 
an additional network which would receive information 
from the program source which would also be part of the 
existing telephone system. The system architecture 4 0 
will then be utilized to provide for enhanced multimedia 
information through psychographic manipulations or other 
enhancements to the systems to provide for an improved 
interactive multimedia system. 

The system 4 0 as above-mentioned includes an IMM 4 2 
that provides much of the optimization of the program 
source material for presentation to MCPS 44. 

To more specifically describe this optimization 
technique, refer now to Figure 3 which is a block diagram 
of an optimization method in accordance with the present 
invention. This optimization technique has been 

described in detail in U.S. Patent Application Number 

f entitled, "Method for the Production and 

Transmission of Enhanced Multimedia Information.", having 
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a filing date of 

t assigned to the assignee in the 

present invention, and that patent application .is 
incorporated by reference in this application. The 
following paragraphs along with the accompanying figures 
will provide the details regarding the optimization 
method and how it will be used advantageously to provide 
an enhanced interactive multimedia system. 

The purpose of the IMM 42 is to provide maximum 
interactivity while at the same time providing maximum 
retention of the program model. It is also important 
that there be minimum transit time for the interactivity 
while the information has maximum replication. 
Therefore, it is very important that the program model 
psychographic parameters be well described. For example, 
the spatial, color, temporal, audio response, material 
concept, contention perception all should be very well 
described and defined in the program model. 

Referring now to Figure 3 what is shown is a first 
embodiment of a flow chart for providing an enhanced 
interactive image that utilizes the principles of the 
present invention. The flow chart 100 comprises the 
steps of providing a program model to a separator. The 
separator 102 will divide the information into primary 
and secondary layers of interactive multimedia 
information. The separation is automatic and can be 
accomplished in a variety of ways. For example, the 
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layers can be separated by production sources. In 
another example, separation can be accomplished through 
key coding the layers. In yet a third example, .the 
layers can be spatially separated or separated by the 
various colors. Finally, layers of information could be 
separated by a filtering process. 

The primary layers are provided to the compression 
generation block 104. There are a variety of ways that 
the multimedia data can be changed or generated to use 
less bandwidth. For example, compression algorithms or 
their equivalents could be utilized to reduce the 
bandwidth used. In addition generators, such as in a 
tone generator system, could be utilized to reduce the 
bandwidth required. Finally, key coding systems could be 
utilized to reduce bandwidth use. Such systems will be 
discussed in more detail later in the specification. 

In this embodiment, the primary layer is provided to 
an encoder where the primary layer is prepared for 
transmission (block 108) . The primary layer is then 
decompressed (block 110) . Thereafter the primary layer 
of information is decompressed (block 110) . The primary 
layer is then decoded and mixed with the secondary layer 
of data information (block 112) to provide an enhanced 
interactive multimedia image (motion, video or other) or 
data to a display. 

Similarly, the secondary layer is compressed through 
block 114, encoded (block 116) and then transmitted to 
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block 118 to decompress and mix block 110. The two 
signals (primary and secondary) are then sent to display 
112. 

In this embodiment, for example, in the optimization 
of video images, the primary layer can be the foreground 
image, the secondary layer can be background information. 
Through the use of this type of optimization technique 
multimedia information can be enhanced while at the same 
time utilizing significantly less bandwidth. 

To more fully understand this feature refer now to 
the following discussion. In a typical interactive 
multimedia system the information is all sent along one 
layer. The information that can then be transmitted is 
limited by the bandwidth of that layer. 

In the prior art, the interactive multimedia 
information that could be transmitted along typical 
networks or transmission paths that are very limited 
because, for example, in the case of vide images the 
bandwidth is not adequate to provide a high quality image 
to a display. 

Hence, in the present invention, by separating the 
multimedia information into primary and secondary layers 
and thereafter compressing the more important information 
utilizing well known compression algorithms, a system is 
described that can produce enhanced interactive 
multimedia information that easily be transmitted over 
existing networks. 
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To more fully describe the psychographic enhancement 
feature of the present invention refer now to Figure 4 
which shown the various possibilities from a particular 
program model. The program model is provided to the 
5 separator 42 of the multimedia system* 

Psychographic enhancements are critical to the 
improvement in interactive multimedia transmission and 
reception enhancements in the context of the present 
application is information that is not transmitted but 
10 operates on, operates in conjunction with, or operates as 

a supplement to transmitted multimedia information. 
There are three separate categories that will be 
described that relate to psychographic enhancements. 
The first category will be described as a cross 
15 correlation between the information that is being 

transmitted and being enhanced due to the presence of 
information that is not transmitted. Dithering of image 
is an example of this in that dithering masks artifacts 
of images that are present and that improves the image 
20 quality. This type of system does not remove the 

artifacts but actually just masks imperfections. A 
second example in the audio arena where secondary audio 
materials such as a sound of an ocean or the like which 
might mask problems in the audio quality of the primary 
25 sound (voice, music or the like) . 

The second category is where the signal is actually 
changed without the use of any control signal; for 
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example, through interpolation or some other technique* 
The typical example of that is a graphic equalizer in 
which certain frequencies are enhanced depending on the 
range of the particular device. Another example of the 
second category is to frequency or amplitude compress a 
certain signal so as to further enhance the frequencies 
that are being transmitted. It is also known to use 
various filters to sharpen or provide certain information 
that will actually modify the signal without controlling 
it per se. 

Finally, the third category is using the primary and 
secondary information to drive the other generators that 
might be present within the multimedia system. This can 
be utilized to either enhance the multimedia information 
or enhance the program model. An example of this is the 
use of real-time graphics frequency spectrum displays to 
enhance a music juke box type of program model. 

As is seen in Figure 4, the primary multimedia 
information layer can be compressed to reduce the 
bandwidth utilizing well known algorithms. It is also 
seen that the signal can be replaced by a generator that 
is responsive to the primary/ secondary layers signals. 
Finally, a key code could be used to cause information to 
be provided from a look-up table or the like. 

Although all of the above methods provide advantages 
in accordance with the present invention, key coding has 
some additional non-obvious advantages when utilized in 
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the optimization system of the present invention. In the 
following paragraphs the use of various key coding 
systems will be described generally along with, their 
attendant advantages. 

Typically, when looking at an interactive multimedia 
information signal there are several components of that 
information. The first component is the data or the 
multimedia information itself that is being conveyed. 
The second component is referred to as program model 
dynamics. That is the changes that occur in the 
interactive multimedia information due to for example, a 
fade that allows for a transition from one scene in the 
graphics or video image to another. Conversely, if you 
want to wipe away an image there is information 
associated with the multimedia data that would call out 
for that transition to change efficiently. 

Finally, the third category of interactive 
multimedia information is what will be referred to in 
this specification which will allow a particular device 
or system to go from one category to another. In a 
typical interactive multimedia information system all 
this information is required to adequately transmit such 
information . 

In its simplest form, a key has an identifiable code 
which dictates the commands on the other side of the 
device. The clearest example of such a keying system 
would be the very simple dual tone mult i- frequency (DTMF) 
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signal. This type of signal can be used in the 
telecommunications area to provide keying for low 
bandwidth protocol. These keys would then command a code 
table on the side of the network to provide certain 
information about the multimedia information to be 
displayed without requiring actual transmission of the 
multimedia information . 

A more specific version of this type of key coding 
is what will be referred to in this specification as 
control information keying. What is meant by controlled 
information keying is where a key code is utilized to 
access particular types of commands which can then be 
used to control other items on the other side of the 
network . 

Such a table would then be utilized to access a 
certain set of multimedia information in the network. A 
final version of key coding will be called program 
branching keying is described by each of the keys 
representing a certain branch identification. Thus in 
this type of key coding the key is cross referenced to a 
particular branch of the interactive multimedia program 
where each of the branches allows plurality of functions 
or commands to be accessed in order to replicate the 

program model. 

The important feature that is provided by all of 
these types of keying coding arrangements is that 
information already present on the network can be 
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utilized. Therefore, the processing power inherent in 
the network or the system being accessed can be utilized 
rather than having to have to provide that processing 
power within the optimization system itself. 

It is also important to develop means to improve the 
transmission quality of the multimedia information, for 
example, the information may be transmitted utilizing a 
typical transmission algorithm such a CCITT v. 32, or Bell 
202 with standard communication file data transfer 
protocols. The interactive multimedia information could 
also utilize specialized protocols that are optimized for 
the particular interactive multimedia information that is 
to be transmitted. In so doing the algorithm for the 
compression algorithm can be interactively matrixed with 
the transmission algorithm to provide the highest quality 
information with the maximum interactivity with the 
minimum transmission line. 

Referring now to Figure 5, what is shown is a flow 
chart that shows the cooperation of the transmission 
algorithm with the compression algorithm to produce a 
high quality multimedia image. The flow chart comprises 
providing a program model in which the primary and 
secondary layers are separated (block 202). The primary 
layer is compressed and encoded (blocks 204 and 206) . 

A control element (block 210) is utilized to control 
a compression matrix and a transmission matrix. 

These two matrices comprise a plurality of compression 
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algorithm and transmission algorithm respectively that 
are interactively controlled such that as the various 
algorithms are detected the quality of the multimedia 
information and the speed of the transmission are 
interactively determined. 

The quality of the information could be determined 
manually or through the use of some control circuitry. 
It should be understood that these same matrices could 
also be used on the secondary layer. It was not shown or 
described for purposes of clarity and simplicity. 

Referring now to Figure 6 what is shown is a block 
representation of a matrix of compression algorithm with 
transmission algorithm that could be utilized in 
accordance with the present invention. The circles 302 
aligned in the vertical direction are the compression 
algorithms. The rectangles 3 04 aligned in the horizontal 
direction are the transmission algorithms. 

For example the compression algorithms could be 
JPEG, a generator with MIDI, and a key for a weather map 
background. Similarly, the transmission algorithms could 
be optimized for JPEG, data compression for MIDI, or DTMF 
for key transmission type algorithms. To provide the 
highest quality multimedia information while at the same 
time utilizing minimum bandwidth the different algorithms 
can be selected in an interactive manner. 

Hence, a first compression algorithm could be 
selected along with the first transmission algorithm. 
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The multimedia information is reviewed either for image 
or audio quality than a second compression algorithm is 
selected. The multimedia information is reviewed and if 
the quality is not acceptable then a second transmission 
algorithm is selected. The quality of the information is 
reviewed. This process is repeated until the highest or 
desired quality multimedia information and interactivity 
speed are provided. 

The multimedia information derived from the 
compression/transmission algorithms can be analog or 
digital in nature. However, in a digital signal there 
are certain other features that can be taken to advantage 
that can be utilized in accordance with the present 
invention. 

It is known that digital data information is 
typically sent in a file which specifies certain 
parameters of that data and the data information itself 
and within the data information itself is information 
which may not change for a certain set of files. In the 
case of an image file, the header information may specify 
the dimensions, pixel depth, and certain other features 
of the particular image. This file may take up a much as 
twenty percent of the data file. 

Conversely, in a file such as MIDI music file which 
comprise a plurality or a series of music notes, the 
header may include instrument information, related 
information and characteristics of that particular file. 
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In both of the above mentioned examples, the header 
information may not change, through the use of the 
optimization method the amount of information may be 
significantly reduced over time. 

Hence, in the case of the image file f the header 
could be sent first with no compression or with lossless 
data compression as the secondary file because it will 
always remain the same. The date file itself can then be 
compressed down to its smaller size. 

Another method for enhancing the psychographic 
parameters is to provide some form of error detection and 
adjustment. As has been mentioned before the detection 
and adjustment can be accomplished via interpolation of 
the error. An alternative method of error corrector is 
through an error correction/transmission algorithm. What 
is meant by this, is relating the transmission to the 
compression to enhance interactivity. 

In this type of system before the file is sent the 
base file is compressed and then decompressed. This 
decompressed file is called an expanded compressed base 
file. The expanded compressed base file is then compared 
to the original base file and an error file is then 
developed (the error file being the difference between 
the base file and the expanded compressed base file) . 
The error file is compressed and sent along with the 
compressed version of the base file down the line. These 
files are then combined and decompressed to provide an 
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enhanced image. 

Referring now to Figure 7 it is seen that a data 
file utilizing this technique could initially be 
separated into primary and secondary layers. The primary 
layer could be compressed using a first compression 
algorithm, the header could be sent first along a first 
transmission path and the compression signal could be 
sent along a second transmission path. 

Therefore, the amount of storage necessary for the 
file is significantly reduced through secondary 
compression techniques. This information can then be 
transmitted or stored across the network rather than 
having to have all that information stored within a 
particular device within the optimization system. 

It should be understood by one of ordinary skill in 
the art will recognize that the number of algorithms is 
not limited to the number shown in the figures. In 
addition it should be recognized that the order or the 
selection of the algorithms could be changed and that 
would be within the spirit and scope of the present 
invention. 

The present invention has been discussed in terms of 
compressing the primary layer or layer and by compressing 
and transmitting that primary layer in a particular way 
the interactivity of the system is enhanced. It should 
be understood that it may be equally important to enhance 
secondary layers to produce the same effect. 
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Therefore, it may be important to enhance the 
secondary layer, it may be important to enhance the 
primary layer or it may be important to enhance both. 
Therefore, the present invention through the use of 
5 compression and transmission algorithms and through the 

psychographic enhancement of the program model can 
enhance interactivity of a multimedia system. 

It should also be understood that the function of 
the compression and transmission algorithms can also be 
10 done through other means; for example, a signal generator 

could be used to provide the same information. That is, 
a signal generator responsive to a particular layer or 
layer of information could be utilized to provide that 
information or some level of information that is 
15 representative of that layer. For example, a tone 

generator responsive to a signal from the secondary layer 
to provide the tone that would be representative of that 
secondary layer . 

Conversely, some type of graphics generator could be 
20 utilized to respond to that same type of signal to 

provide a certain type of graphic image in a video 
system. Finally, it should also be understood that the 
psychographic parameters can be adjusted by human 
operator or in the alternative can be adjusted or 
25 modified by an automatic means. 

The optimized multimedia information from the 
program source material is then processed by the MCPS 44 
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(Figure 2) to be utilized interactively with IMDs 52. 
Referring now to Figure 8 what is shown is the preferred 
embodiment of a MCPS 44. The MCPS comprises a. 
distributed computing architecture 500. The distributed 
computing architecture 400 includes a master node 502 
that has several server nodes 504 which in turn are 
connected to several ports 506. When connected via the 
existing networks such as to a switched telephone network 
and then on to connection to the IMD 52 which has its own 
processing storage and computing structure the entire 
network can be operated as a massive distributed 
computing environment. This environment shares all 
dimensions of computing, storage, transmission and 
peripheral resources (printing, product ordering, mailing 
functions, etc.). This type of computing architecture 
would include dynamic port allocation and would include 
incremental failure characteristics to allow for 
robustness of the MCPS 44. 

It also is very important in this system 
architecture to have an interactive multimedia device 52 
which will allow for the receipt of high quality 
multimedia information from the system architecture. The 
IMD 52 can be a printer that has been attached to a video 
display. It can also be a facsimile machine which could 
be utilized to received enhanced facsimile-like 
information. The IMD 52 can also be utilized with a 
television monitor to provide enhanced audio, video and 
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graphic information thereto. Other connections to common 
household devices such as a VCR and household heating 
control systems can also be utilized by the IMD.52. 

Figure 9 is a preferred embodiment of an interactive 
multimedia decoder (IMD) 52 A. The IMD 52 A comprises 
several components. The telephone line is coupled to a 
multimedia modem 522. The multimedia modem is coupled to 
a multimedia memory 524 which can be an expandable 
dynamic random access memory (DRAM) 524. The multimedia 
modem chip 522 provides data to a multimedia decompressor 
device 526. The multimedia memory provides data to 
graphics /character generator 536, speech generator 538 
and music synthesizer 540. 

In addition, the output of the generators 536, 538 
and 540 are provided to a video control chip 532. Video 
control 532 provides signals to a standard television 
display and receives signals from a standard television 
source. The multimedia modem 532, the multimedia memory 
524, the multimedia decompressor 526, the multimedia 
digital/audio control 528, the video control 532 and 
music synthesizer 544 are all ultimately controlled by an 
interactive control interface 53 0 which manages the 
operation of all of the above elements. The video 
control 528 is coupled to a standard telephone keypad 
input or for a television remote-type device or a special 
IMD remote can be utilized in a variety of ways which 
will be discussed in detail hereinafter. 
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Personalized and demographic information (such as 
the age, race, sex and other personal characteristics) of 
the user along with the technical information associated 
with the IMD 52 (serial no., number of generators, type 
5 etc.)- Upon connecting the IMD 52 to the MCPS 44, the 

IMD 52 can both forward this information either at the 
beginning of the session or any time afterward. The 
information can then be updated through the MCPS 44 or 
directly IMD 52 through selection using a keypad or 
10 remote control or by receiving data from the MCPS 44. 

Another mode of operation is the delivery of 
multimedia during the period when a communication network 
is not in use. In this way, large amounts of multimedia 
information can be efficiently transmitted and stored in 

15 the IMD 52 for 

IMD for later review and enhanced interactivity. 

The function of each of the different components in 
a preferred embodiment is described in a summary fashion 
below. 

20 Multimedia Modem 522 

a. Responsible for all communications between standard 
phone line, optional serial port, interface to multimedia 
memory, multimedia decode, audio control, and processor 
control modules. 

25 B. Supports standards protocol for half -duplex, full 

duplex, and half-duplex high speed operation. 
C. On-chip encode/decode capability, D/A, A/D for 
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voice, facsimile, and data functions. 

D. Dual tone multi-frequency (DTMF) detect and 
generation. c * 

E. Auto-detect voice/ facsimile/data switch for 

5 transparent mode transition. 

F. Incorporates controller unit with binary file 
transfer, facsimile, data, and voice modes, and optional 
proprietary multimedia processor control optimized 
protocol firmware. 

10 G. Firmware allows IMD to use multimedia modem to 

perform call processing function including telephone call 
dialing and connection, unattended receipt of data and 
fax among other functions. 
Multimedia Memory 524 

15 a. Nominal DRAM or VRAM for image mixing/processing , 

and auxiliary multimedia data store. 

B. Nominal ROM for resident IMD control program. 

C. Optional co-resident DRAM for multimedia data store 
and program/data store. 

20 D. Optional non-volatile storage (extendible) . 

E. Memory control unit for VRAM/ROM/DRAM and non- 
volatile storage. 
Multimedia Decode 526 

a. Responsible for real-time decompression of images 
25 transferred to or stored in the IMD 52 with multiple 

algorithm coding. 

B. On chip inverse discrete cosine transform processor. 
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C. Reverse quantizer decoder /tables. 

D. Built-in zoom, pan, chroma key, mix from compressed 
data incorporates interfaces to video data bus , . 
multimedia memory, multimedia modem, video control, and 
microprocessor control sections. 

Video Control 532 

a. Responsible for all IMD 52 video mixing, 
enhancements, and display functions. 

B. Pixel processor for mix, zoom, pan, chroma key, 
transform on pixel data, transitions. 

C. Graphics processor for figures (e.g., rectangles 
with color fill) generation, sprites, text with foreign 
characters, and scrolling. 

D. Digital to analog conversion, analog to NTSC, NTSC 
video plus stereo audio to RF . 

Graphics /Character, Speech Generator. Mus ic Synthesizer 
536. 538 and 540 

a. Responsible for enhancing received analog/digital 
audio, music synthesis generation, and overall analog 
mixing and audio effects. 

B. Incorporates decoding burden. 

C. Sampled instrument synthesis from compressed MIDI 
input . 

D. Built-in micro-controller for multi-task generation. 

E. Dual analog source mix, digital audio and 
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synthesizer mix, analog audio control (volume, bass, 
treble, balance) for output to analog left/right audio* 
Interactive Multi-Task Processor 530 

A* Responsible for multi-task execution of resident and 
5 downloaded IMD code for operation in conjunction or 

independently of the MCPS. 

B. Master/ slave microcontroller architecture for multi- 
task control of communications, multimedia memory, 
multimedia decode, digital video control, digital 

10 audio/ synthesis, and interface management. 

In a preferred embodiment, the IMD 52 (Figure 2) 
will be utilized with a television monitor to transmit 
and receive multimedia information. 

Figures 10, 11 and, 12 show different embodiments of 

15 an IMD 52 . Figure 10 shows an IMD 52B which has an 

output to a printer 501 and a telephone handset 540. The 
printer 501 then is utilized to print multimedia 
information responsive to the telephone handset. 

Figure 11 shows an IMD 52C that has an output to a 

20 television 700 and will print facsimile-like information 

received from the system architecture 40. This so-called 
FAX IMD 52C includes a facsimile machine internal 
thereto . 

Figure 12 shows an IMD 52D that has an output to a 
25 television 53 and also has an output connected to a fax 

machine. This will allow for a viewer to view the 
incoming facsimile messages and other multimedia 
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information and then print them via a fax machine. 

Yet another preferred embodiment of the present 
invention comprises a MCPS 44 which incorporates a 
standard voice response computer or interactive voice 
response system or the like to perform digital or analog 
\ voice functions of the MCPS 44. In addition this 
\ apparatus would also perform the DTMF functions for MCPS 
program control. To perform the multimedia information 
transmission to the IMD 52 and to control the switching 
between voice data , fax the system would utilize a 
modification to standard voice response systems, 
interactive voice response system or a modification 
thereof. By utilizing the appropriate IMD 52 protocol 
reconnection to a standard telephone through the above 
mentioned modified systems. Accordingly, either one 
standard telephone line or a plurality of telephone lines 
operating in parallel may be utilized for transmission 
from the MCPS 4 4 to the IMD 52. 

Another critical feature of the IMD is to have a 
remote control that will work in conjunction with the TV 
or other display or the like to provide enhanced 
multimedia information. To more fully explain this 
feature refer now to Figure 13. The remote control 900 
looks much like a telephone keypad. It has the numerals 
(0-9) and symbols (*-#) that are part of an ordinary 
telephone keypad. It includes an enter key 902 that is 
typically utilized to change information or change 
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channels in the case of a television set. The control 
900 would also include a volume key 904 and a channel or 
memory select key 906 , a connect key 908 and a multimedia 
toggle button 910. It could also include a credit card 
slot 912. This credit card slot 912 would be utilized by 
the viewer to allow for the purchase of certain items 
directly while viewing the television screen. 
Finally, the control 900 includes special effect keys 
916, for example, for allowing for the browsing of a 
multimedia directory while simultaneously displaying a 
picture in the picture of the current broadcast TV 
channel . 

The remote control 900 could utilize a radio 
frequency signal or audio signal to interact with the 
receiver and/ or IMD 52 for the control of the IMD, 
control the selection of multimedia information, and for 
the control of other household devices- It is also known 
that a more conventional remote control could be utilized 
such as one that controls a VCR or a television and its 
use would be within the spirit and scope of the present 
invention. 

Another feature of the above-identified system 
architecture 4 0 of Figure 2 is that the program 
information can be linked to the network to provide for 
enhanced interactivity and program quality. For example, 
the program source can follow or be synchronized with the 
cable or broadcast feed to provide for enhanced 
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distribution of program information. One specific 
example of this to take a new program, like CNN 
broadcasting, which consists of several short news items. 
For each news there could be stored large archives or 
program material related to a particular story. Hence, 
through multimedia interaction with for, example, the 
remote control 900 the stored program material can be 
accessed. This would allow the user to review the 
material in whatever detail was desired. 

Another example, is in an advertising feed if a 
particular item was advertised, there would be the 
ability to refer to more detailed information about the 
product . 

Yet another example of such a system is to have one 
service linked to another. An advertisement could be 
linked to a means for ordering the particular product 
being advertised including the placement of a phone call 
by the 

IMD to the desired telephone number. In so doing, the 
linked system allows for additional services to be 
accessed. 

The important requirements for this linked feature 
is that there is a knowledge of the contents of the 
system program requirements and there is a knowledge of 
where the user is in the program. For example, an IMD 
could be used for channel program identification in which 
the telephone call is linked to a particular channel that 
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has been selected. Another example is link demographics 
for targeted interactive advertising, hence , in an 
advertisement for a diaper, for example, there would be 
the facility to access advertisements for related items 
such as baby powder, baby oil or the like through the use 
of this linked approach. 

It should be understood that the IMD 52 itself could 
be utilized as a call processing system. Finally, it 
should be understood that there could be multiple MCPS 4 2 
to provide for very large scale call processing. 

Although the present invention has been described in 
accordance with the embodiments shown in the figures, one 
of ordinary skill in the art recognizes there could be 
variations to the embodiments and those variations would 
be within the spirit and scope of the present invention. 
Accordingly, many modifications may be made by one of 
ordinary skills in the art without departing from the 
spirit and scope of present invention, the scope of which 
is defined solely by the appended claims. 
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WHAT IS CLAIMED IS: 

1. An interactive multimedia device for use in a 

telephone network comprising: 

a multimedia modem means for receiving interactive 
5 multimedia information from the telephone network; 

memory means for receiving interactive multimedia 
information from the multimedia modem means; 

means for compressing at least a portion of the 
interactive multimedia information received from the 
10 multimedia modem means and the memory means; 

means for decompressing at least a portion of the 
interactive multimedia information; 

means for transmitting the uncompressed portion of 
the interactive multimedia information received from the 
15 telephone network; 

mixing the decompressed portion with the 
uncompressed portion and providing an output; 

means for providing the output of the interactive 
multimedia information to a display; and 

means for interactively controlling the multimedia 
modem means, compressing means, decompressing means, 
mixing means and providing means to provide interactive 
multimedia information to the display. 

2. The device of Claim 1 in which the modem means 
is a voice activated device that receives the voice 



20 



25 



information. 

3. The device of Claim 1 in which the interactive 
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controlling means comprises a microprocessor. 
* 4. The device of Claim 1 in which the mixing means 

comprises means for receiving the compressed signal and 
the uncompressed signal. 
5 5. The device of Claim 1 which further comprises 

means for dual tone multi- frequency detection (DTMF) from 
a network • 

6. An interactive multimedia device for use in a 
telephone network comprising: 
10 means for receiving interactive multimedia from the 

telephone network; 

memory means coupled to the receiving means for 
storing the interactive multimedia information; 

means for compressing a first portion of the 
15 interactive multimedia information; 

means for transmitting the first portion along a 
first transmission path; 

means for transmitting a second portion of the 
interactive multimedia information along a second 
20 transmission path, the second portion of the interactive 

multimedia information being uncompressed; 

means coupled to the transmitting means for decoding 
the first portion of the interactive multimedia 
information; 

25 means coupled to the decoding means and the second 

portion transmitting means for mixing the decoded first 
portion and the uncompressed second portion; and 
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means coupled to the mixing means for printing the 
interactive multimedia information received from the 
mixing means; and means for interactively controlling the 
receiving means , the memory means, the compressing means, 
the first transmitting means, the second transmitting 
means, the decoding means, the mixing means and the 
printing means to print multimedia information. 

7. A communications network for interactive 
multimedia transmission comprising: 

interactive multimedia mastering system means for 
receiving program materials from a program source; 

multimedia call processing system responsive to 
interactive multimedia mastering system program materials 
for providing information; 

means interactively responsive to the interactive 
multimedia mastering system means and the multimedia call 
processing system for controlling the flow of multimedia 
information to the multimedia call processing system; and 

a plurality of interactive multimedia devices (IMD) 
for receiving and transmitting multimedia information to 
and from the multimedia call processing system. 

8. The system of Claim 7 in which the multimedia 
call processing system further comprises: 

means for creating multiple multimedia files for use 
by the multimedia mastering system. 

9. The system of Claim 7 in which the interactive 
multimedia devices are coupled to a printer to print 
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information from the multimedia network. 

10. The system of Claim 9 in which the interactive 
multimedia devices are coupled to a display. 

10(a). The system of Claim 9 in which each of the 
interactive multimedia device has the ability to store 
information related to demographics and the 
characteristics of the interactive multimedia device and 
thereafter forward the information at an appropriate 
time. 

10(b). The system of Claim 10(a) in which 
interactive multimedia device can delay transmission of 
information stored there within until the communication 
network is not in use. 

11. The system of Claim 10 in which the display 
comprises a television. 

12 ♦ The system of Claim 11 in which control of the 
transmission of program source material is accomplished 
using a standard voice response system. 

13. The system of Claim 11 in which control of the 
transmission of program source material is accomplished 
using an interactive voice response system. 

14 . The system of Claim 11 in which control of the 
transmission of program source material is accomplished 
using a computer processing system. 

15. The system of Claim 12 in which a printer is 
connected to the IMD. 

16. The system of claim 12 in which a facsimile 
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machine is connected to the IMD. 

17. The system of claim 11 in which a telephone 
handset is used to control and select the transmission 
and presentation of program source material. 

5 18. The system of claim 11 in which telephone keypad 

is used to control and select the transmission and 
presentation of program source material. 

18. The system of claim 11 in which a remote control 
is used to control and select the transmission and 

10 presentation of program source material. 

19. The system of claim 12 in which the IMD 
comprises a music synthesizer and a multimedia 
decompression means for generating program source 
material. 

15 20. The system of claim 12 in which the IMD 

comprises a music synthesizer and a key coding means for 
generating program source material. 

21. The system of claim 12 in which the IMD 
comprises a music synthesizer and a generator means for 

20 generating program source material. 

22. The system of claim 12 in which utilizes 
switching between data, voice and facsimile on a single 
telephone line to transmit program source material. 

23. The system of claim 12 in which utilizes 

25 switching between data, voice and facsimile on multiple 

telephone lines to transmit program source material in 
parallel. 
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22. The system of claim 12 in which certain portions 
are designated as primary program material portions and 
certain portions of the MGPS are designated as. secondary 
program material portions and are used to communicate 
5 primary and secondary multimedia information. 

25. The system of claim 24 in which program 
transmission can occur simultaneously. 

26. The system of claim 24 in which program 
transmission can occur at a specified period of the day. 

10 27. The system of claim 24 in which the program 

transmission can occur when the telephone line is not 
being utilized. 

28. An interactive multimedia device for use in a 
telephone network comprising: 
15 a multimedia modem means for receiving 

communications from the telephone network; 

a multimedia memory means coupled to the multimedia 
modem means; 

a multimedia decode means coupled to the multimedia 
20 memory means; 

a digital video means coupled to receive data from 
the multimedia decode means; 

a synthesis control means for enhancing the 
information received from the multimedia modem means; 
25 a microprocessor for controlling the multimedia 

modem means, the multimedia memory means, the multimedia 
decode means, 
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the digital video means and the synthesis control 
means for providing enhanced interactive multimedia 
information. 

29. The interactive multimedia device of Claim 28 
5 in which the multimedia modem means further comprises: 

means for switching between voice, data, and 
facsimile. 

30. The interactive multimedia device of Claim 29 
in which the multimedia modem includes means for 

10 performing call processing functions. 

31. The interactive multimedia device of Claim 29 
in which the multimedia modem includes means for 
automatically placing telephone calls. 

32. The interactive multimedia device of Claim 29 
15 in which the multimedia modem means further comprises: 

means for dual tone multi-frequency detection and 
generation. 

33. The interactive multimedia device of Claim 32 
in which the multimedia modem means further comprises: 

20 means for transparently switching between facsimile, 

voice and data. 

34. An interactive multimedia device for use in a 
telephone network comprising: 

multimedia modem means for selecting interactive 
25 multimedia information; the multimedia modem means 

further comprising; 

means for separating the multimedia into primary and 
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secondary layers based upon a program model; 

memory means for receiving interactive multimedia 
information from the multimedia modem means; 

means for enhancing the primary layer from the 
multimedia modem means in accordance with the program 
model to enhance interactivity; 

means for decompressing the primary layer of the 
interactive multimedia information; 

means for transmitting the uncompressed portion of 
the interactive multimedia information received from the 
telephone network; 

mixing the decompressed portion with the 
uncompressed portion and providing an output; 

means for providing the output of the interactive 
multimedia information to a display; and 

means for interactively controlling the multimedia 
modem means, compressing means, decompressing means, 
mixing means and providing means to provide interactive 
multimedia information to the display. 

35. The device in accordance with Claim 34 in which 
the multimedia modem means further comprises: 

means for providing the secondary layers through 
alternate psychographic parameters; and 

means for combining the primary and secondary 
layers . 

36. The device of Claim 34 in which the enhancing 
means : 
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means for encoding the primary layer in accordance 
with the first compression algorithm. 

37. The device of Claim 34 in which the enhancing 
means comprises: 

means for encoding the primary layer in accordance 
with a first transmission algorithm. 

38. The device of Claim 34 in which the enhancing 
means comprises: 

first means for encoding the primary layers in 
accordance with a plurality of transmission algorithms; 

second means for encoding the primary layers in 
accordance with a plurality of compression algorithms; 

means for determining the perceived quality of the 
multimedia information responsive to said first and 
second encoding means; and 

means for alternately selecting between the first 
and second encoding means to provide an acceptable 
perceived quality of multimedia information. 

39. The device of Claim 34 in which the separating 
means comprises keying the primary and secondary layers 
to the psychographic parameters. 

40. The device of Claim 35 in which the separating 
means comprises separate production sources for providing 
the primary and secondary layers. 

41. The device of Claim 34 in which the separating 
means comprises means for spatially separating the 
primary and secondary layers in accordance with the 
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psychographic parameters, 

42. An interactive multimedia device for use in a 
telephone network comprising: 

multimedia modem means for selecting interactive 
multimedia information; the multimedia modem means 
further comprising; 

means for keying multimedia information into primary 
and secondary layers based upon a program model; 

memory means for receiving interactive multimedia 
information from the multimedia modem means; 

means for enhancing the primary layer from the 
multimedia modem means in accordance with the program 
model to enhance inactivity; 

means for decompressing the primary layer of the 
interactive multimedia information; 

means for transmitting the uncompressed portion of 
the interactive multimedia information received from the 
telephone network; 

mixing the decompressed portion with the 
uncompressed portion and providing an output; 

means for providing the output of the interactive 
multimedia information to a display; and 

means for interactively controlling the multimedia 
modem means, compressing means, decompressing means, 
mixing means, and providing means to provide interactive 
multimedia information to the display. 

43. A system for interactively providing enhanced 
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information related to a communication network, the 
network including a first set of program source materials 
that is produced for a user, the network further 
including a second set of program source material that is 
5 related to the first set of program source material, the 

second set of program source material is stored in a 
separate location than the first set of program source 
material, the system comprising: 

means for linking the first set of program source 
10 material with the second set of program source material; 

and 

means for interactively producing the second set of 
program source material; wherein the user of the system 
can produce additional information related to the first 
15 set of program source material. 

44. The system of Claim 43 including means for 
simultaneously displaying the second set of program 
source material with the first set of program source 
material* 

20 45. The system of claim 43 in which the second set 

of program source materials provides more detailed 
information related to the first set of program source 
materials. 

46. The system of claim 43 in which the second set 
25 of program source materials provides information that is 

different from but related to the first set of program 
source materials. 
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47. The system of claim 43 in which the 
interactively producing means interactively displays the 
second set of program source material while the first set 
of program source material is produced. 

48. The system of claim 43 which further comprises 
means for selecting related services based upon the 
second set of program source material. 
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